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ABSTRACT 

Multinational  Experiment  3  was  the  third  event  in  a  series  of  United  States  Joint  Forces 
Command  (USJFCOM)  multinational  experiments.  Multinational  Experiment  3  was  a 
process-refinement  experiment  whose  goal  was  to  build  on  the  lessons  learned  from 
Multinational  Limited  Objective  Experiments  I  and  11,  and  to  continue  exploring  concepts 
and  supporting  tools  for  effects-based  planning.  Results  will  assist  the  development  of 
future  processes,  organizations,  and  technologies  at  the  operational  and  joint  task  force 
level  of  command.  Additionally,  Multinational  Experiment  3  provided  the  participating 
nations  an  opportunity  to  examine  issues  associated  with  operational  net  assessment. 
Coalition  Interagency  Coordination  Group,  coalition  intelligence,  surveillance,  and 
reconnaissance,  multinational  information  sharing,  logistics,  coalition  based  health  services 
support,  information  operations,  and  knowledge  management.  The  North  Atlantic  Treaty 
Organization  (NATO)  also  examined  concepts  associated  with  their  NATO  Response  Force. 
This  paper  will  highlight  the  experiment  background,  design,  objectives,  assessment 
concept,  analysis  construct,  analysis  organization,  and  some  of  the  key  analytical  findings 
and  lessons  learned  from  the  experiment. 
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EXPERIMENT  DESIGN 

As  several  nations  had  already  developed  their  own  variations  of  an  EBP  concept  prior  to 
MNE  3,  it  was  necessary  to  write  a  new  version  of  the  EBP  concept  specifically  for  MNE  3. 
This  version  was  based  on  features  from  the  national  EBP  concepts  and  provided  a  common 
baseline  for  MNE  3. 

To  examine  the  viability  of  and  the  procedures  required  for  implementing  EBP,  certain 
overarching  and  supporting  concepts  were  required  to  accurately  depict  the  planning 
environment.  These  included:  ONA,  CIE,  Coalition  Interagency  Coordination  Group 
(CIACG),  Coalition  Intelligence,  Surveillance  and  Reconnaissance  (CISR),  MNIS, 
Logistics,  Coalition  Based  Health  Services  Support  (CBHSS),  Information  Operations  (10), 
and  Knowledge  Management  (KM). 

The  United  States  chose  to  implement  a  distributed  Coalition  Task  Force  Headquarters 
(CTFHQ)  based  on  the  Standing  Joint  Force  Headquarters  (SJFHQ)  organizational 
construct  linked  by  a  wide  area  network  (WAN).  The  Combined  Federated  Battle 
Laboratories  (CFBL)  Network  formed  the  backbone  of  the  WAN.  All  partner  nations 
provided  personnel  to  man  the  CTFHQ.  The  CTF  Command  group  and  functional  leads 
were  physically  located  in  the  USJFCOM  Distributed  Continuous  Experiment  Environment 
Lab  (DCEE).  All  other  participants  were  physically  located  in  their  home  nation  facilities. 
The  experiment  was  designed  so  that  NATO  and  Allied  Command  Transformation  (ACT) 
would  examine  and  implement  the  EBP  process  in  parallel  with  the  CTFHQ.  The  NATO 
Response  Force  (NRF)  headquarters  was  largely  co-located. 


Specific  participation  facility  locations  included: 


Participant 

Location 

Australia 

1.  Defence  Science  and  Technology  Organization,  Fern  Hill  Park,  Canberra, 
ACT 

Canada 

1.  Canadian  Forces  Experimentation  Centre,  Shirley’s  Bay  Detachment, 

Ottawa 

France 

1.  CIADIOS,  Tavemy  AB  (BA  921),  Taverny 

Germany 

1.  Bundeswehr  Military  Intelligence  Center  (BMIC)  Grafschaft-Gelsdorf 

2.  Bundeswehr  Center  for  Analyses  and  Studies  -  OR  Division,  Ottobrann 

3.  Bundeswehr  Operations  Command,  Potsdam 

4.  Bundeswehr  ADP  -  Support  Center,  Euskirchen 

United  Kingdom 

1 .  Defence  Science  Technology  Laboratory  Portsdown  West  Facility, 

Fareham  Hampshire 

United  States 

1.  Distributed  Continuous  Experiment  Environment,  USJFC0M/J9,  115  Lake 
View  Parkway,  Suffolk,  VA 

2.  Joint  Battle  Center,  1 16  Lake  View  Parkway,  Suffolk,  VA 

3.  Defense  Information  Systems  Agency  AITS/JPO,  Arlington,  VA 

NATO 

1.  Castlegate,  Germany 
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Figure  2:  MNE  3  Construct 

During  the  planning  of  the  experiment,  there  were  design  considerations  that  would  influence 
the  conduct  of  the  experiment  and  the  results. 

□  Experiment  goals  were  ambitious  due  to  the  first  time  integration  of  a  highly 
complex  process,  organization,  and  new  technology. 

□  The  experiment  used  8  hours  of  operational  play  per  day  instead  of  24  hours. 

□  MNE  3  was  a  single  trial  experiment;  however,  it  is  just  one  piece  of  a  larger 
experimentation  plan  conducted  by  USJFCOM  concerning  these  concepts. 

□  Participant  training  and  knowledge  of  the  concepts  was  less  than  envisaged  upon 
fielding  of  the  concepts. 

□  Training  and  rehearsal  intended  to  occur  during  the  experiment  validation  event 
(Rock  Drill)  did  not  happen  as  planned.  Instead  the  time  was  used  to  complete  the 
development  of  tactics,  techniques,  and  procedures  (TTP)  for  the  process. 

□  Component  and  higher-level  command  play  was  limited  to  white  cell  responses. 

During  experiment  execution,  limitations  emerged  that  influenced  experiment  play  and 
subsequent  analysis. 

□  Disconnects  and  gaps  between  steps  in  the  planning  process  were  not  adequately 
resolved  prior  to  experiment  start. 

□  TTP  stated  what  needed  to  be  done  but  not  how  to  complete  tasks. 

□  It  was  unclear  how  the  supporting  concepts  would  be  integrated  into  the  planning 
process. 

□  Players  came  to  the  experiment  with  different  perceptions  of  what  a  command-led 
process  should  be. 

□  Players  had  to  spend  time  during  Week  0,  intended  for  a  walk-through  of  the 
process,  for  team-building  and  concept  of  operations  briefings. 

□  Players  were  too  inexperienced  with  the  organizational  construct  and  daily  battle 
rhythm,  which  impacted  negatively  on  the  conduct  of  the  EBP  process. 

□  Players  had  no  prior  exposure  to  the  EBP  planning  tool  before  Week  0. 
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■  Does  the  Coalition-Based  Health  Services  Support  process  support  EBP? 

EBP  depends  on  a  complex  OTay  of  processes.  These  processes  involve  the  integration  of 
many  concepts  into  one  coordinated  endeavor.  If  these  concepts  are  employed  to  their  full 
extent,  and  each  contributes  necessary  information  to  EBP,  the  CTF  and  NRF  will  be 
successful  in  performing  EBP.  Successful  EBP  performance  is  defined  by  the  ability  to 
meet  the  six  requirements  identified  in  proposition  1. 

The  MNE  3  EBP  concept  describes  a  13 -step  process  that  begins  with  the  identification,  in 
the  Commander’s  Initial  Guidance,  of  key  effects  to  be  considered,  develops  a  detailed 
assessment  of  the  numerous  effects  and  actions  that  can  be  taken  against  nodes  needed  to 
achieve  these  key  effects,  and  examines  the  resources  available  to  be  used  to  accomplish 
these  actions.  It  then  develops  a  single  course  of  action  (COA),  where  considered  military 
and  possibly  nonmilitary  actions  are  then  synchronized.  The  result  of  this  is  a  single  plan 
reflected  in  the  ETO.  The  steps  are  essentially  sequential  but  some  parts  of  the  process  may 
be  conducted  in  parallel,  and  the  complete  process  would  require  several  iterations  to  refine 
the  overall  plan  for  EBO. 

Objective  2:  Develop  and  assess  organizations  to  support  coalition  and  NRF  EBP. 
Proposition  2:  The  EBP  organizational  design  will: 

□  Enable  the  flow  of  information 

□  Facilitate  the  generation  of  knowledge 

□  Enhance  planning 

□  Improve  decision  making  and 

□  Produce  an  effective  ETO. 
oCOIs 

■  What  organizational  structure  is  required  for  EBP? 

■  What  behaviors  and  competencies  are  required  for  EBP? 

■  What  mitigating  human  factors  impact  upon  EBP? 

The  CTFHQ  and  XDJTFHQ  organizational  structures  identified  for  MNE  3  were  based 
upon  the  design  of  cross-functional  teams  connected  in  a  habitual  way  to  distributed  experts 
including  nonmilitary  government  and  civilian  agencies  and  coalition  partners.  The  staff 
was  organized  to  enable  the  effective  flow  and  integration  of  information.  It  was  expected 
that  the  elimination  of  functional  “stovepipes”  would  reduce  coordination  time  and  allow 
synergistic  planning  and  execution.  The  fluid  movement  of  information  between  people  via 
machine  interfaces  was  both  a  challenge  and  an  opportunity  for  commanders.  It  was 
proposed  that  if  managed  properly,  this  organizational  structure  would  produce  better 
decisions  faster  and  ultimately,  the  output  of  the  EBP  process,  the  ETO,  would  produce  the 
desired  effects  when  executed. 

Objective  3:  Identify  technology  requirements  to  support  coalition  and  NRF  EBP. 
Proposition  3:  Technology  will  augment  the  human  ability  to  conduct  EBP  through  a  suite 
of  tools. 

oCOI 
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Objective  3  Methodology.  Qualitative  and  quantitative  data  were  used  to  assess  MNE  3 
implemented  technologies,  and  to  identify  functionality  requirements  for  EBP.  The  aim 
was  to  identify  technology  requirements  to  support  coalition  and  NRF  EBP. 

Furthermore,  an  experiment  analysis  workshop  was  convened  to  enable  all  analysts  to 
contribute  their  inputs  to  the  final  report.  Partners  discussed  their  insights  into  the 
objectives  and  concepts  for  which  they  had  lead  analysis  responsibility,  as  well  as  proposed 
experiment  findings. 

Figure  3  depicts  the  sequence  of  events  in  the  analysis  and  reporting  process. 


Analysis 

Plan 


■  Sailor  Cbncept.Deyelopers 


"n  r  7\ 


Feedback  during  game  play 


7  V  \  / 


Event  Play  &  Players 
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SehiorJ. 
rte'ader. 
|semin1ar’ 


Executive 
Report  . 


Figure  3:  MNE  3  Analysis  Construct 
ANALYSIS  ORGANIZATION 

The  analysis  team  was  organized  to  support  the  analysis  functions  of  the  experiment: 
assessment  planning,  data  collection,  data  analysis,  and  reporting.  All  partner-nation 
analysts  were  completely  integrated  into  the  analysis  team.  The  responsibility  to  lead  the 
analysis  effort  for  specific  objectives  and  concepts  was  divided  amongst  the  partner 
analysis  teams. 

The  hierarchy  of  data  flow  and  organization  for  the  analyst  team  and  observers  is  depicted 
in  Figure  4. 
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PROCESS  MODELING 


Conceptual  models  representing  the  functional  and  temporal  aspects  of  EBP  and  supporting 
processes,  organizations,  and  technologies  were  developed  using  the  G2  and  C3TRACE 
process  model  tools.  These  models  captured  internal  and  external  tasks,  processes, 
organization,  and  communications  played  during  the  experiment.  The  models  were 
developed  during  the  experiment  validation  Rock  Drill  and  the  experiment.  Figure  6 
depicts  the  components  of  the  conceptual  models. 


Functional  Component: 

a.  Process/task  analysis  (EBP  CONOPS) 

b.  Organizational  structure  analysis  (CTFHQ) 

c.  Technology  support/information  requirements 
(CIE) 

Temporal  Component : 

a.  Situational  understanding  over  time 
(intent,  information,  knowledge) 

b.  Organization  structure  over  time 
(collaboration/teaming) 

c.  Interactions  over  time 
(communications  between  groups, 
man/machine  interface) 

Relational  Component : 

a.  Dependencies  between  key 
and  organizational  elements 

b. ldentification  of  strengths  and 
weaknesses  (leverage  points),  and 
feedback  within  EBP  system 

EBP  Concept  &  CONOPS 


Figure  6:  Conceptual  Model  Components 

The  output  of  these  models  can  then  be  compared  to  the  physical  model  of  the  processes, 
organizations,  and  technologies  employed  by  the  CTFHQ  as  well  as  the  XDJTFHQ  during 
experiment  execution  to  build  a  relational  understanding  of  key  processes  and 
organizational  elements.  These  measurements  and  observations  are  then  used  to  update 
conceptual  models  so  as  to  document  the  developed  EBP  process  for  subsequent 
experimentation. 
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The  SCDs  also  made  ten  high-level  observations  arising  from  the  experiment’s  in-focus 
sessions,  participant  seminars,  and  AARs.  The  following  paragraphs  describe  these 
observations. 

Multinational  Ejfects-Based  Operations 

The  process,  organization  and  technologies  used  in  MNE  3  require  further  experimental 
refinement  before  they  are  ready  for  operational  use.  SCDs  offered  the  following 
observations: 

□  Effects-based  operations  are  not  necessarily  new,  but  encourage  a  new  way  of 
thinking;  help  to  focus  on  the  strategic  aim 

□  Effects-based  thinking  gives  a  broader  range  of  options  for  the  employment  of 
military  and  other  interagency  actions 

□  EBO  provides  a  coherent  Diplomatic,  Information,  Military,  Economic  (DIME) 
planning  process 

□  EBO  causes  a  more  holistic  view  of  the  key  actors  and  adversaries  (political, 
military,  economic,  social,  infrastructure,  information) 

Coalition  Operational  Net  Assessment  (Knowledge  Base) 

□  A  common  ONA  is  an  essential  enabler  of  coalition  operations 

□  The  utility  of  the  coalition  ONA  is  dependent  upon  nations  being  able  to  share 
relevant  data 

□  National  ONAs  must  be  interoperable,  able  to  be  merged  when  a  coalition  forms 

□  The  ONA  must  incorporate  expert  judgments  of  the  adversary,  potential  spoilers, 
the  operational  environment,  and  ourselves  -  Red/Blue  wargaming 

□  A  visualization  tool  used  to  display  effects  and  relationships  contained  within  the 
ONA  will  help  to  convey  commander’s  intent  and  situational  awareness  to  all  levels 

The  Networked  Coalition 

□  To  effectively  network,  a  coalition  requires  the  highest  possible  degree  of 
information  sharing 

□  A  networked  coalition  employing  EBO: 

o  Facilitates  mission-oriented  operations  by  involved  government  agencies  and 
cooperation  from  non  government  agencies 
o  Contributes  to  mutual  understanding  and  confidence  amongst  the  coalition 
partners 

oHas  the  ability  to  tailor  Diplomatic,  Information,  Military,  Economic  (DIME) 
effects,  actions  and  assess  outcomes  more  precisely  and  more  rapidly 
oWill  be  more  effective  when  there  is  a  mixture  of  face-to-face,  virtual 
collaboration  and  prior  common  training 

Harmonize  Nonmilitary  Capabilities  of  Governments 

□  A  robust  CIACG  proved  essential;  however,  we  need  to  clearly  define  roles  and 
procedures  for  the  CIACG 

□  Incorporating  nonmilitary  capabilities  of  government  into  the  early  stages  of  the 
planning  process,  including  ONA  development,  is  essential 
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□  Training  that  focuses  on  the  challenges  of  Stability  Operations  e.g.  individual  skills 
and  decentralized  operations 

Coalition  Deployment,  Employment,  and  Sustainment 

□  Coalition  EBO  may  require  national  logistics  organizations  to  adapt 

□  EBO  requires  the  coalition  commander  to  have  some  limited  logistics  authority  over 
the  whole  force 

□  Logistics  planning  must  be  fully  integrated  from  the  start  of  the  effects-based 
planning  process 

□  EBO  underscores  the  need  to  plan  for  the  timely  deployment,  sustainment  and 
protection  of  the  civilian  capacity  to  create  desired  effects 

Leader  Competencies 

Leadership  in  a  coalition  collaborative  information  environment  requires  somewhat 
different  skills  than  today’s  commeind  and  control  environment  (information  age  vice 
industrial  age).  Among  the  issues  that  will  require  a  different  approach: 

□  Coping  with  cultural  and  doctrinal  differences 

□  Establishing  trust  and  confidence 

□  Building  habitual  relationships 

□  Exploiting  speed  of  developing  knowledge  and  decision-making 

□  Ensuring  comprehensive  information  sharing  and  timely  dissemination 

□  Mastering  information  overload 

□  Partnering  with  civilian  authorities 

□  Crafting  and  communicating  Commander’s  intent 


Key  participant  findings  from  MNE  3  were: 

□  The  EBP  concept  has  the  potential  to  make  the  CTF/NRF  a  more  effective 
instrument  of  power.  However,  the  EBP  concept  developed  for  MNE  3  is  not 
operationally  mature  and  requires  further  refinement 

□  Players  stated  that  the  best  features  of  the  EBP  process  were: 

o  it  forced  military  planners  to  think  in  terms  of  effects,  which  expanded  alternative 
ways  to  achieve  objectives  beyond  military  actions,  and 
o  collaboration  brought  out  the  best  ideas  from  a  collective  thought  process 

□  Players  stated  the  most  difficult  parts  of  the  MNE  3  EBP  process  were: 
o  the  complexity  of  the  process  inhibiting  thought  and  analysis 

o  understanding  the  process 
o  confusing  terminology 
o  lack  of  an  integrated  tool  suite,  and 

o  particular  difficulty  implementing  the  course  of  action/wargaming  step 

□  There  is  a  need  to  create  a  coalition  logistics  structure  and  plan  as  a  coalition,  not  as 
group  of  individual  nations 

□  The  CIACG  brings,  a  valuable  civilian  perspective  to  military  planners,  the  CTF 
staff  and  the  command  group  that  is  essential  to  an  effective  EBP  process 

□  Contributions  from  subject  matter  experts  such  as  CIACG  and  information 
operations  need  to  be  integrated  in  the  ONA 


15 


Assessed  as 

J ;  "Correct” 

Assessed  as 
“IncorrecC 

Actually 

Correct  . 

HIT 

(error)  ,  . 

!  •  ■  -i-J 
O 

2 

fc 

o 

o 

3 

.  u 

,  FALSE  ALARM 

(error) 

CORRECT  REJECTION 

Figure  7:  Interpretation  of  Responses  within  the  Signal  Detection  Paradigm 

This  provides  a  performance  measure  but  also  gives  an  insight  into  the  bias  of  the 
participants.  For  example,  it  is  possible  to  ascertain  whether  groups  or  individuals  are  more 
likely  to  agree  with  false  information  or  reject  true  information. 

The  understanding  and  probe  statements  were  administered  to  the  participants  on  4 
occasions  at  significant  stages  of  process  which  required  good  common  intent  for  EBP  to 
work.  The  responses  were  examined  by  staff  role,  functional  groups  and  physical  location. 

The  UK  analysis  team  took  the  lead  in  the  development  and  analysis  of  the  common  intent 
probes.  A  brief  sununary  of  their  conclusions  follows. 

□  Overall  it  would  appear  that  the  commander’s  intent  was  not  effectively  dispersed 
beyond  the  command  team,  CIACG  and  plans  team.  This  conclusion  should  be 
tempered  with  the  caveat  that  these  data  represent  only  the  very  first  steps  in  trying 
to  quantitatively  measure  the  commander’s  intent  and  they  were  collected  in  an 
experimental  environment  using  a  coalition  type  command  structure. 

□  The  measurement  methodology  employed  appears  to  measure  the  promulgation  of 
commander’s  intent  across  different  sites.  The  method  appears  to  capture  sensible 
findings,  such  as  the  command  team  has  a  better  understanding  of  the  command 
intent  than  the  rest  of  the  CTFHQ.  Collecting  good  data  in  a  “field  experiment”  is 
extremely  difficult  and  this  method  appears  to  be  able  to  collect  robust  data  without 
undue  interference  with  the  HQ  staff. 

□  It  would  appear  that  the  promulgation  of  commander’s  intent  is  influenced  by 
organizational  proximity  to  the  commander  and  not  geographic  proximity  to  the 
commander.  This  is  supported  by  the  U.S.  site  not  having  a  greater  understanding 
of  commander’s  intent  than  other  nations  but  those  in  the  command  team 
(organizationally  and  geographically  close  to  the  commander)  did  have  a  good 
understanding  of  commander’s  intent.  The  plans  team,  who  were  geographically 
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Multinational  experimentation  continues  to  be  a  critical  element  of  USJFCOM’s  joint 
concept  development  and  experimentation  program.  The  body  of  data  collected  during  the 
multinational  experimentation  series,  as  well  as  insights  from  an  even  larger  body  of 
USJFCOM  and  multinational  experiments,  will  guide  MNE  4.  Each  event  in  the  chain 
brings  us  closer  to  fielding  the  SJFHQ,  to  delivering  innovation  to  the  warfighter,  and  to 
recommending  actions  to  senior  leaders  based  on  experimental  findings. 

Ultimately,  the  multinational  experimentation  program  will  result  in  better  coalition  warfare 
when  the  United  States  and  its  partners  around  the  world  apply  their  military  forces. 
Dissimilar  capabilities  and  perspectives  must  not  hinder  the  ability  to  work  together  in 
combined  military  operations  to  address  complex  international  issues.  By  working  together 
in  a  dedicated  multinational  experimentation  program,  the  United  States  and  its  allies 
ensure  that  they  experiment  as  they  fight. 
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WAN  -  Wide  Area  Network 

XDJTFHQ  -  NATO  Experimental  Deployable  Joint  Task  Force  Headquarters 
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